In the world of virtualization infrastructure, the term, “live migration” refers to the migration of a virtual machine (VM) from a source host computer to a destination host computer. Each host computer (also referred to herein as “host”) is a physical machine that may reside in a common datacenter or in distinct datacenters. On each host, virtualization software includes hardware resource management software, which allocates physical resources to run VMs on the host and emulation software which provides instances of virtual hardware devices, such as CPUs, storage devices, network devices, etc., that are interacted with by the guest system software (i.e., the software executing “within” each VM). Virtualization software running on each host also cooperates to perform the live migration of the VM.
Exemplary implementations of live migration of VMs are described in detail in U.S. Pat. No. 7,484,208 which issued on Jan. 27, 2009, and U.S. Pat. No. 7,680,919, which issued on Mar. 16, 2010. These two patents are expressly herein incorporated by reference in their entirety. In general terms, one important aspect of performing live migration is copying the state of the VM from the source host to the destination host in a manner that allows minimal or insignificant disruption of the VM's execution at the time of the transfer from the source host to the destination host. A challenging component of this state information to be transferred is the contents of the guest physical memory. A VM's guest physical memory comprises those pages of machine memory (i.e., actual physical memory residing in the host) that are mapped or allocated to the VM being migrated. The guest physical memory address space is treated by the guest system software (e.g., the guest operating system) as actual physical memory, but the guest physical memory address space is mapped by the virtualization software to physical pages of machine memory. The main reason it is challenging to transfer the guest physical memory to the destination host in a live migration is that the VM is allowed to continue to execute during the transfer, and thus the VM continues to update the guest physical memory as the guest physical memory is being copied to the destination host.
To copy guest physical memory to a destination host while the VM is executing, an iterative pre-copy scheme may be used, as described in detail in the patents incorporated by reference above. In general, the guest physical memory pages are iteratively copied to the destination host prior to execution of the migrating VM on the destination host. Such iterative copying involves multiple copying operations beginning with copying the entire contents of the guest physical memory from the source host to the destination host, then repeatedly copying the pages dirtied (i.e., written to or modified by the VM) since the previous copy operation. Provided the bandwidth for transmitting the copied pages between the source and destination hosts is high enough, the pre-copy process will eventually converge to a sufficiently small set of guest physical memory pages that can then be successfully copied to the destination host after stunning the VM on the source host, so that the VM can then be resumed on the destination host with minimal or insignificant interruption.
In some situations (e.g., in cases where the pre-copy technique described above cannot converge), the execution of the VM is transferred to the destination host before all of the pages of the guest physical memory of the VM are copied to the destination host. As the VM runs on the destination host and encounters pages that it lacks, but remain present on the source host, the VM demand faults those pages over the network from the source host. This process is called “resume during page-in” (RDPI) and enables the system to guarantee transparent migration success, even for VMs having large working sets of memory pages which have not been pre-copied.
During this period of RDPI, the failure domain of the VM is expanded to include both the source host and the destination host since the VM's memory exists on both the source host and the destination host. As a result, if either of the source host or the destination host crashes during RDPI, the VM needs to be terminated. The network connection between the source host and the destination host will also be an additional source of failure.